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6ESRL  Technical  Research  Reports  and  Technical  Research  Notes  are  intended  for 
sponsors  of  RliD  tasks  and  other  research  and  military  agencies.  Any  findings 
ready  for  implementation  at  the  time  of  publication  are  presented  in  the  latter  part 
of  the  Brief.  Upon  completion  of  a  major  phase  of  the  task,  formal  recommenda¬ 
tions  for  official  action  normally  are  conveyed  to  appropriate  military  agencies 
by  briefing  or  Disposition  Form. 


FOREWORD 


Technological  advancement*  have  led  to  increased  apeed,  mobility,  and  destructive 
power  of  military  operations.  To  permit  commanders  to  make  tactical  decisiona  consis¬ 
tent  with  rapid  change  and  auccession  of  events,  information  on  military  operations  must 
be  processed  and  used  more  effectively  than  ever  before.  To  meet  this  need,  the  Army  is 
developing  automated  systems  for  receipt,  processing,  storage,  retrieval,  and  display  of 
different  types  and  vast  amounts  of  military  data.  There  is  a  concomitant  requirement  for 
research  to  determine  how  human  abilities  can  be  utilized  to  enable  the  command  informa¬ 
tion  procesaing  systems  to  function  with  maximum  effectiveness. 

One  objective  of  the  COMMAND  SYSTEMS  Task  is  to  provide  research  information  by 
which  decision  making  and  information  assimilation  from  displays  may  be  facilitated.  To 
this  end,  studies  are  conducted  on  such  information  presentation  factors  as  amount,  den¬ 
sity,  format,  color  coding,  specificity-generality,  alpha-numeric  vs  symbolic  displays,  rate 
and  degree  of  updating,  probability  data  and  certitude,  individual  and  groip  displays, and 
the  relative  utility  of  different  aensory  and  diaplay  modalities.  The  present  publication 
reports  on  a  portion  of  Subtask  e,  "Effective  aids  in  the  decision  process,"  and  explores 
relationships  between  the  accuracy  with  which  information  is  assimilated  and  the  certi¬ 
tude  that  the  information  assimilated  is  correct. 

The  entire  research  task  is  responsive  to  requirements  of  RDT&E  Project  2JQ24701A* 
723,  "Human  Performance  in  Military  Systems,”  FY  1968  Work  Program,  and  to  special 
requirements  of  the  Combat  Development*  Command  end  the  Army  Materiel  Command. 


yy  E.  UHLANER,  Director 
U.  S.  Army  Behavioral  Science 
Research  Laboratory 


RELATION  OF  CERTITUDE  JUDGMENTS  TO  CHARACTERISTICS  OF  UPDATED 
SYMBOLIC  INFORMATION 


BRIEF 


Requirement: 

In  the  Army's  command  information  processing  systems,  updating  of  situational  date  adds  to  the 
difficulty  of  information  assimilation  and  may  affect  confidence  in  accuracy  of  information  and,  conse¬ 
quently,  the  quality  of  decisions  made  on  the  baeia  of  the  information.  The  purpoee  of  the  present  re¬ 
search  was  to  anplore  the  affects  of  type  and  number  of  updates,  total  amount  of  Information  presented, 
and  ael acted  conap icuity  coding  techniques  on  confidence  and  the  relationship  of  confidence  to  accuracy 
of  information  assimilation. 


Procedure: 

Forty-eight  subjects  ware  presented  successive  pairs  of  slfdas.  The  first  slide  of  a  pair  contained 
12,  II,  or  24  military  flag  symbols  randomly  positioned  on  a  map.  The  second  slide  was  identical  to  the 
first  except  that  2,  4,  or  6  symbols  had  bean  added,  removed,  or  repositioned.  After  the  subjects  had 
successively  viewed  bodt  slides  of  a  pair,  the  second  elide  was  removed.  Subjects  than  indicated  on  a 
paper  print  of  the  first  slide  all  ipdates  they  hod  noted  in  the  second  slide.  Performance  was  measured 
in  terms  of  aocurscy  of  assimilation  (a  percentage  score)  and  confidence  In  that  accuracy  (ranging  from 
1 -absolutely  uncertain,  to l-ebeolutely  certain).  The  research  design  was  replicated  over  three  enhance¬ 
ment  tschniquea-single  cue  coding,  double  cue  coding,and  hard  copy,  as  well  as  with  no  enhancement. 


Findings: 

Increasing  either  amount  of  information  presented  or  amount  of  ipdating  resulted  in  a  decline  in  both 
mean  accuracy  and  mean  certitude,  the  rate  varying  widely  over  the  different  enhancement  techniques 

aaM  M«ea  Stekwe  wF  umMeiw 

BnO  OVBt  Vfpm  OT  IqJQsvV* 

Mean  certitude  paralleled  mean  accuracy  reasonably  well,  but  the  accuracy-certitude  correspondence 
did  not  hold  for  individual  performance  scores.  Over  all  conditions,  only  38  pore  ant  of  the  certitude 
variance  could  be  accounted  for  by  accuracy  variance. 

The  more  effective  the  enhancement  technique,  the  higher  the  certitude-accuracy  relationship.  With 
the  bast  enhanesment  technique  (doublecue  coding),  84  percent  of  the  certitude  variance  could  be 
accounted  for  by  accuracy  variance,  with  the  poorest  (hard  copy) ,  only  30  percent. 

There  was  evidence  of  both  over-cortitudo  and  undar-cartitude,  over-certitude  tending  to  increase 
with  the  leas  effective  enhancement  techniques. 


Utilisation  of  Findings: 

Findings  suggest  need  to  improve  agreement  between  a  man’s  performance  in  information  assimila¬ 
tion  and  his  estimate  of  that  performance. 

In  the  absence  of  specific  information  to  the  contrary,  it  should  be  assumed  that  most  techniques 
which  aid  assimilation  will  also  improve  certitude  estimates  though  the  effect  may  vary  with  the  tech¬ 
nique. 

Reliance  on  group  estimates  should  bo  encouraged,  at  least  until  techniques  for  improving  estimation 
are  devised. 

Determination  of  the  role  of  certitude  in  the  timeliness  and  appropriateness  of  decisions  should  bo  a 
prime  research  objective.  To  this  end,  certitude  should  be  measured  in  the  same  metric  as  accuracy 
(percent) . 
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RELATION  OF  CERTITUDE  JUDGMENTS  TO  CHARACTERISTICS 
OF  UPDATED  SYMBOLIC  INFORMATION 

OBJECTIVES 


The  Cammed  Systems  Task  of  tbs  U.  S.  Amy  Behavioral  Sclsnes 
Resssreb  laboratory  Is  conduct  Inc  »  ssrlss  of  projects  in  which  a  variety 
of  display  variables  are  systematically  investigated  in  tenm  of  tbelr 
effects  on  information  assimilation  and  decision  making  in  a  coemand 
and  control  settinf.  A  concoaltant  of  information  assimilation  is  the 
confidence  or  feeling  of  certitude  the  viewer  has  in  the  accuracy  of  bis 
assimilation.  To  the  extent  that  feelings  of  confidence  or  certitude 
appropriately  reflect  the  assimilation  performance  upon  which  they  are 
presumably  based,  decision  behavior  could  be  enhanced.  It  is  therefore 
of  Interest  to  know  more  about  the  certitude-accuracy  relationship  as  a 
function  of  various  display  parameters  and  to  determine  the  degree  to 
which  accuracy  and  certitude  may  be  differentially  affected. 

In  an  earlier  study^tbe  nudber  of  symbols  removed  in  an  updatli* 

(2  through  8)  had  negligible  effect  on  accuracy  of  assimilation,  but  the 
greater  ntsdber  of  ohanges  did  result  in  an  appreciable  decrement  in  certi¬ 
tude.  The  present  experiment  was  an  attempt  to  determine  whether  the 
findings  for  tvdating  generalise  beyond  the  specific  oiretamtanees  of 
the  earlier  study  and  whether  related  variables  show  similar  effects. 

The  specific  objectives  of  the  present  study  were: 

1.  To  determine  whether  accuracy  of  information  assimilation  from 
displays  and  judpmnts  of  certitude  are  similarly  affected  by  (l)  anoint 
of  information  presented,  (2)  type  of  updating  o hangs,  and  (3)  extant  of 
updating  change. 

2.  To  determine  the  extent  to  which  the  acouraoy-oertltude  relation¬ 
ship  varies  with  techniques  to  enhance  the  eonsplcuity  of  updates— updated 
hardcopy  for  reference,  single-cue  codii*,  doUble-cu»  codii*. 


METHOD 


Experimental  Design 

The  Independent  variables  investigated  were  total  mnfcer  of  sywbols 
in  a  slide— 12,  18,  2k  (amount);  type  of  chains  introduced  in  a  sligle 
update— symbols  removed,  added,  repositioned;  extent  of  chains  in  a  singls 


^ Andrews,  R.  8.,  and  R ingel,  3.  Certitude  Judgments  and  the  accuracy 
of  information  assimilation  in  visual  displays.  U.  8.  Army  Behavioral 
Science  Research  Laboratory  Teobnloal  Research  Bote  IV3.  May  196k. 


update—  2,  k t  $  symbols  changed,  and  enhancement  techniques— hard  copy, 
single-cue  coding,  double -cue  coding.  A  no-coding  condition  was  also 
imposed  to  provide  a  bass  psrformncs  index.  Nsasurss  of  accuracy  of 
assimilation  and  certitude  About  that  accuracy  ware  coordinate  dependent 
variables. 

Tbe  basic  design  of  tbs  experlasnt  is  sbown  sebseatleally  in 
Figure  1. 


Figure  1.  Schematic  design  of  experiment  for  a  single  enhancement  technique 


There  were  four  replicates  of  tbe  basic  design,  one  for  each  enhancement 
technique.  Twelve  subjects  were  randomly  assigned  to  each  replicate  and 
each  subject  received  all  treatment  coeblnatlons  of  anoint  by  extent  by 
type.  Within  each  replioate ,  tbe  twelve  subjects  were  randomly  divided 
into  three  groups  of  four  subjects  each.  Groups  differed  only  in  order 
of  presentation  of  type  of  change  and  the  randosdsed  sequence  of  slide 


presentation  vitbin  type  of  change.  Thus,  within  each  replicate  each 
type  change  was  administered  first,  second,  and  third,  and  there  were 
nine  different  random  sequences  of  slides. 


Subjects 

Forty-eight  ailltary  personnel  of  above  average  intelligence  (General 
Technical  Aptitude  Area  score* 'of  HO  or  higher)  with  noxml  or  corrected 
nornal  visual  acuity  participated  as  subjects.  Military  sophistication 
or  tralniiy,  not  being  pertinent  to  performance  in  the  experimental  task, 
was  not  a  requirement.  Sacb  subject  participated  for  the  90  to  120  nin- 
utes  required  to  complete  a  session. 

Stimulus  Presentation 

Stimuli  were  in  tbs  fora  of  39sm  negative  transparencies  depicting 
a  number  of  military  flag  symbols  positioned  on  a  skeleton  nap.  Bach 
pair  of  slides  was  node  up  of  a  basic  slide  and  an  updated  slide  which 
differed  from  the  basic  slide  only  in  the  changes  introduced  la  updating. 
Figures  2  and  3  show  a  pair  of  such  slides.  The  illustrations  differ 
from  the  slides  used  in  the  experiment  only  in  that  the  projected  image 
gave  white  symbols  and  lines  on  a  dark  hadgrowd. 

Positioning  the  symbols  on  the  basic  slides  and  selecting  the  symbols 
to  change  in  updating  were  both  done  by  a  randomisation  process  with  only 
one  restraint:  There  were  four  different  military  unit  spools— Infantry, 
Artillery,  Air  Defense,  and  Engineer— on  each  basic  slide  in  the  approxi¬ 
mate  proportions  of  1/2,  l/k,  1/6,  and  1/12,  respectively.  In  no  instance 
of  the  spools  "removed"  condition  were  all  of  any  particular  type  of 
unit  removed. 

Figure  k  shows  the  coding  used  for  sirgle-cue  and  dovfcle-cue  updates. 
For  the  other  enhancement  technique,  hard  copy,  a  page  print  of  the  first 
slide  was  made  available  to  the  subject  simultaneously  with  the  unooded 
updated  slide.  Ubcoded  symbols  were  identical  to  single  cue  except  that 
there  was  no  "arrow"  for  moved,  no  "V"  for  added,  and  no  "broken  line 
pTiWTir  ror  xwtn • 

subjects  were  seated  directly  in  front  of  and  19  feet  away  from  the 
screen  on  which  the  slides  were  rear  projected  with  an  image  area  approx¬ 
imately  7.6  feet  by  9  feet.  AAlent  illumination  was  maintained  at  a 
level  sufficient  to  reduce  the  possibility  of  after  images,  to  provide 
light  for  the  men  to  read  and  mark  their  answer  sheets,  and  yet  not  high 
enough  to  interfere  with  easy  viewing  of  the  displayed  material. 


Based  on  the  Vexbal  and  Arlthnwtlc  Reasoning  Tests  of  the  Army  Classi¬ 
fication  Battery. 
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Figure  2.  Basic  slid*  with  12  symbols 
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Figure  4.  Examples  of  single-cue  and  double-cue  coding 


Procedures  and  Instructions 

Before  the  start  of  a  session,  subjects  were  briefed  on  the  nature 
of  the  experiment  and  their  participation.  The  experimenter  read  the 
Instructions  aloud,  while  each  nan  followed  alone  on  a  printed  copy. 

Two  practice  slide  sets  were  presented  and— after  the  experlsmnter  had 
answered  all  questions  and  assured  himself  that  the  subjects  understood 
their  task— the  experiment  proper  began. 

Within  each  pair  of  slides,  a  basic  slide  was  presented  for  60 
seconds;  the  updated  slide  (the  same  as  basic  slide  with  2,  4,  or  6 
symbols  added,  removed,  or  repositioned)  was  then  presented.  Simultane¬ 
ously  with  the  showing  of  the  updated  slid*;,  the  subjects  were  given  a 
question  relating  to  "bow  many"  of  a  particular  type  unit  had  been 
changed.  Forty-five  seconds  were  allowed  for  answering,  after  which  a 
second  question  relating  to  "locations"  of  a  specific  change  was  given 
and  another  b5  seconds  allowed  for  answering  and  continued  viewing  of 
the  updated  slide.  Thus,  the  subjects  had  a  total  of  90  seconds'  view¬ 
ing  time  for  the  updated  slide.  The  updated  slide  was  then  turned  off, 
and  the  subject  turned  to  his  response  sheet  (replloa  of  basic  slide) 
and  Indicated  all  units  which  he  thought  had  been  changed  as  a  result  of 
the  updating.  These  responses  provided  a  measure  of  Information  assimi¬ 
lation.  The  subject  bad  60  seconds  to  complete  his  answer  and  lndloate 
his  certitude  for  that  answer  on  the  scale  provided.  The  same  procedure 
was  followed  until  nine  pairs  of  slides  had  been  presented,  subjects 
were  then  permitted  a  five-minute  break.  This  procedure  was  followed 
for  each  of  the  three  type-of-change  sequences  within  a  session. 
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Dependent  Variables 


The  major  dependent  variable  eoneldered  was  the  certitude  expressed 
by  the  subjects  In  the  accuracy  of  their  information  assimilation.  To 
indicate  his  certitude,  a  subject  circled  the  phrase  on  the  following 
scale  which  best  described  bow  certain  or  uncertain  he  felt  in  the  accu¬ 
racy  of  his  response.  Sue cei sire  integers  asslgnsd  to  the  eight  cate¬ 
gories  of  the  eontlnuun  were  the  values  used  in  the  analyses. 


Absolutely 

Very 

Moderately 

Slightly 

Slightly 

Moderately 

Very 

Absolutely 

Uncertain 

Uncertain 

Uncertain 

Uncertain 

Certain 

Certain 

Certain 

Certain 

The  performance  accuracy  score  was  computed  as  follows: 

Rltbt  _ _  x  ioo 

Amber  Aigbt  +  Errors  of  Coadsslon  +  Errors  of  Omission 

An  error  of  commission  was  made  when  the  subject  marked  a  symbol  that  had 
not  been  changed,  an  error  of  omission  when  he  failed  to  nark  a  symbol 
that  had  been  changed. 


RESULTS 

A  sunary  of  the  analysis  of  variance  performed  on  the  certitude 
scoree  is  shown  in  Table  1.  All  four  main  effects  and  three  of  the  inter¬ 
actions  were  significant  beyond  the  .001  level.  Only  one  of  the  other 
effects  aehleved  significance  at  or  beyond  the  .09  level.  These  flndligs 
conform  closely  to  those  from  the  analysis  of  the  accuracy  data^  in  a 
prior  study  in  which  the  identical  variables  and  interactions  were  the 
only  ones  achieving  statistical  significance  at  or  beyond  the  .09  level 
except  for  the  accuracy  by  type  of  change  by  enhancement  technique  inter¬ 
action  which  was  not  significant  in  the  accuracy  analysis.  The  relation¬ 
ships  between  mean  certitude  and  mean  accuracy  for  each  of  the  main  effects 
are  plotted  in  Figures  9  through  8.  It  is  apparent  from  these  figures 
that  the  certitude-accuracy  relationship  typically  approximates  linearity 
except  for  enhancement  technique  for  which  the  relationship  departs 
appreciably  from  linearity  (Figure  8). 


*'Tbe  accuracy  data  and  analyses  used  for  comparison  purposes  wvre 
extracted  from  a  prior  report  (Vlelno,  F.  L. ,  Andrews,  R.  S. ,  and 
Rlngel,  S.  Consplculty  coding  of  updated  systolic  Information. 

U.  S.  Amy  Behavioral  Science  Research  Laboratory  Technical  Research 
Rote  192.  May  1969)  end  are  not  presented  in  detail  in  the  present 
publication. 
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Certitude 

Figure  5.  Mean  accuracy  end  certitude  for 
amount  of  information 


Certitude 

Figure  6.  Mean  oc curacy  and  certitude  for 
type  of  change 
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Figure  7.  Mean  accuracy  and  certitude  for 
extent  of  change 


Certitude 

Figure  8.  Mean  accuracy  and  certitude  for 
enhancement  technique 
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Pursuing  tbs  similarities  fart  bar,  tbs  Newaan-Ksuls  Student lied 
rang*  tsstr'was  applied  for  tasting  differences  between  pairs  of  Mans 
within  each  of  tbs  sain  variables.  The  results  are  shown  in  Table  2. 

Only  for  enhancement  technique  did  certitude  and  accuracy  scores  differ 
la  terns  of  which  pairs  of  tans  eaecapassed  statistically  significant 
differences.  For  all  tin  effects,  including  enbanesnent  technique, 
the  rank  orders  of  tbs  tat  were  identical.  It  is  likely  that  the  non¬ 
linearity  of  the  certitude  •accuracy  relationship  and  the  greater  dif¬ 
ferences  among  enbanostnt  techniques  for  accuracy  tans  than  for  confi¬ 
dence  tat  were  In  large  part  a  function  of  the  fact  that  these  compari¬ 
sons  were  between  groups,  l.e. ,  each  subject  perfomsd  on  only  a  single 
snhanoatnt  technique.  It  was  to  be  expected  that  an  Individual's  certi¬ 
tude  estlaatee  would  be  less  precise  in  an  absolute  sets  than  in  a 
relative  senae.  Thus,  a  constant  error  In  terms  of  a  subject's  subjective 
certitude  scale  would  affect  between  subject  comparisons  but  not  within 
subject  comparisons. 


Table  2 

MIAX  CBtrnUDB  FOR  LEVKD3  WZTBZI  SACK  NADI  VARIABLE 


Variables 


Condition  Levels 


NOTf  Voluoa  connected  by  linos  do  not  diftar  iignificantly  at  P<.0».  intnaa  tn  paranthoaaa  ara  corrc.ponding 
accuracy  aeoraa  and  apoaar  only  for  main  affacta  whara  confid.nca  and  accutacy  dtffar  tn  tarma  of  atgntficanca  of 
dtffarancaa  botwoon  pair,  of  moano. 


Plotting  the  significant  tvo«way  interactions— type  of  c hangs  by 
snhanoenent,  aaount  of  infoxmtlon  by  enhancement,  and  axtant  by  type 
of  ebaaga  helps  clarify  tba  nature  of  the  interactions*  Figure  9  shows 
differential  certitude  affects  for  type  of  change  across  snhann— wt 
techniques,  with  repositioning  tbs  major  contributor.  Since  dmft>le-cue 
coding  produced  the  highest  accuracy  for  tbs  symbols  removed  and  symbols 
added  conditions  and  was  equivalent  to  the  highest  (hard  copy)  for  the 
symbols  repositioned  condition,  the  relatively  lower  certitude  level  for 
the  symbols  repositioned  condition  nay  be  attributable  to  a  sore  realistic 
certitude  estivate  with  double-cut  coding. 
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7-5 
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6.5 
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No  Bard  Single  Double 

Code  Copy  Cue  Cue 

Enhancement  Techniques 


ttjurt  9.  •■icon  cartituHe  for  ty"«  of  dange  by  enhonesment  technique 


Tb*  shape  of  tb*  functions  plotted  In  Fleur*  10  for  tb*  amount  of 
information  by  enhancement  technique  Interaction  *r*  very  similar  to 
tbo**  for  accuracy  «xc«pt  that  tb*  rank  ord*r  for  *nbanet— nt  t*cbnlqu* 
on  accuracy  «u  double  cue,  single  cut,  bard  copy,  and  no  cod*  at  *aob 
amount  level.  Tb*  finding  reported  In  T*bl*  2  of  no  *lgnlflo*nt  dlf- 
f«r*ne*t  In  confldtne*  among  bard  copy,  single-cue,  and  double -cue 
enhancement  aust  b*  qualified  in  light  of  this  amount  by  enhancement 
interaction,  sine*  bard  copy  was  significantly  diff*r*nt  from  tb*  otb*r 
two  at  tb*  24 -amount  level.  Similarly.  tb*r*  was  a  dlff*r*ntlal  ebang* 
in  certitude  in  going  from  tb*  18  to  2a  amount  level  across  *nbanc*msnt 
technique,  tbougb  tb*r*  was  no  significant  dlff*r*ne*  b*tv*«n  tb*  lB  to 
2b  amount  levels  over  all  *nbane*n*nt  t*cbniqu*s. 

Tb*  final  significant  interaction,  *xt*nt  by  typ*  of  ebang*,  is 
plotted  in  Figure  11.  Tb*  certitude  functions  ar*  virtually  identical 
to  tbo**  far  accuracy.  Thus,  tb*  flndli*  of  significant  differences 
among  extents  of  ebang*  as  simple  effects  needs  b*  tempered  merely  by 
tb*  finding  that  tb***  differene**  exist  only  for  tb*  added  and  reposi¬ 
tioned  typ*  of  ebang*— not  for  symbols  removed— and  that  differene** 
among  type*  of  ebang*  do  not  bold  between  symbols  removed  and  symbols 
added  at  tb*  2-  and  4 -extent  of  ebaiqp  levels. 

Tb*  seemingly  blgb  correspondence  between  aeeuraey  and  certitude  In 
terms  of  bow  these  measures  were  affected  by  tb*  variables  of  interest 
pravted  a  eloeer  look  at  tbls  relationship.  Sine*  the  certitude  eetl- 
smtee  were  of  a  qualitative  nature  not  translatable  into  a  metric  directly 
comparable  to  tb*  aeeuraey  scores  (percent  right),  eorrelatlon  teobnlques 
were  applied.  Product  moment  correlation*'  based  on  tbe  324  pairs  of 
measures  (aeeuraey  and  certitude)  within  eaob  enhancement  technique  and 
tbe  1296  pairs  of  all  snhannmnsiit  techniques  combined  produced  coeffi¬ 
cients  of  .79  for  double  cue,  .63  for  single  cue,  .49  for  bard  copy, 

.30  for  no  eode,  and  an  overall  eorrelatlon  of  .62.  At  least  two  Infer¬ 
ences  oan  be  drawn  from  these  data.  Tbe  first  is  that  improved  enhance¬ 
ment  teobnlques  contribute  not  only  to  aeeuraey  of  information  assimilation 
but  also  to  tbe  appropriateness  of  certitude  estimates  regarding  assimila¬ 
tion  performance.  Second,  despite  tbe  similarity  of  tb*  effects  tbe  main 
variables  bad  on  accuracy  and  certitude,  tbe  relationship  on  an  individual 
score  basis  over  all  conditions  is  not  impressive  since  no  more  than 
approximately  36  percent  of  tbe  variance  in  certitude  estimates  can  be 
accounted  for  by  accuracy  variance. 
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Figure  10.  Mean  certitude  for  amount  of  information 
by  enhancement  technique 
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Because  of  the  subjective  nature  of  tb«  certitude  continuum,  it  mi 
not  possible  to  determine  wbetler  certitude  estimates  war*  typiaally 
higher  or  lover  than  warranted  by  tbo  acounaoy.  Onto  can  bo  adduced  for 
both  over-  ond  under-estimation.  Table  3  abovs,  for  enhanceasnt  tech¬ 
nique,  what  porcant  of  tha  100S  eorroot  roaponaaa  bad  oortltudo  estimates 
of  "absolutely  oortain"  aaaoolatod  with  tboa  and  what  porcant  had  oltbar 
"abaolufcaly"  of  “vary  certain ',  aa  wall  aa  tba  converse,  for  example, 
what  parcant  of  roaponaaa  ratad  "absolutely  cartaln"  proved  to  ba  lOOJt 
corraot. 


Tabla  3 

CORRESPONDENCE  BETWEEN  ACCURACY  AND  CERTITUDE 
BY  ENHANCSOQVT  TECHNIQUE 


Enhancement  Techniques 

Cartltuda  Expressions  DoUbla 

Roaponaaa  HX#  right— 
estimated  "abaolutaly 

cartaln"  by  stfeject  23^  62$  67%  7l£ 

Roaponaaa  IOC#  right— 
aatlaatod  "abaolutaly" 

or  "wry  cartaln"  58  85  86  93 

Roaponaaa  aatlaatod 

"abaolutaly  certain"— 

100*  right  33  62  89  94 

Roaponaaa  aatlaatod 

"abaolutaly"  or  "vary 

cartaln"— 100*  right  31  34  82  84 


Tha  figurea  In  tha  first  two  rows  can  ba  viawad  as  evidence  of  under* 
estimation  of  parfomanoa  and  those  In  tha  last  two  rows  as  evldanea  of 
over-estimation,  albeit  tenuous.  Mora  important  la  tha  avldanca  support¬ 
ing  tha  notion  that  enhancement  techniques  which  tend  to  Improve  accuracy 
of  Information  assimilation  also  improve  tba  relationship  between  accuracy 
and  subjective  estimates  of  accuracy.  That  the  relative  ease  of  the 
information  assimilation  task  as  a  function  of  enhancement  technique  does 
affect  certitude  is  further  evidenced  in  Table  4.  The  mean  parcant  right 
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associated  with  sacta  certitude  oategory,  particularly  tbs  higher  ones, 
differs  considerably  as  a  function  of  enhancement.  Thus,  the  effective 
soale  range  for  the  no-code  condition  is  conpressed  downward  fron  the  top 
relative  to  the  dodble-cue  and  single-cue  conditions.  Typloally,  the 
90  percent  correct  point  falls  between  k  and  9  on  the  scale  (slightly 
uncertain  and  slightly  certain). 


TSble  4 

NEAR  PHtCEHT  RIGHT  FOR  EACH  SCALE  CATEGORY 
SEPARATELY  BY  EKHAHCiMEffr  TECHMIQUE 


Certitude 

categories 

Enhancement  Techniques 

Double 

Cue 

Single 

Cue 

Hard 

Copy 

Ho 

Code 

8 

96 

96 

83 

76 

7 

88 

87 

76 

79 

6 

78 

76 

63 

63 

5 

6l 

44 

94 

36 

k 

49 

rf 

45 

36 

3 

44 

6o 

92 

92 

2 

39 

49 

42 

38 

1 

23 

a 

10 

23 

•  Unusually  high  valua  atmbuiabla  to  tha  law  40)  taaponaai  ol  which  ona  waa  100%  and  ona  B3%  corract. 


16 


IMPLICATIONS  OF  THE  FINDINGS 


For  Information  assimilation  tasks  of  tba  typs  considered  hare, 
mean  certitude  function!  appear  to  conform  to  the  mean  accuracy  functions 
reasonably  veil,  at  least  in  terms  of  shape.  Because  of  the  different 
metrics.  It  was  not  feasible  to  determine  quantitatively  the  degree  of 
conformity  or  whether  the  certitude  Judgments  were  predominantly  over¬ 
estimates  or  underestimates.  The  conformity  of  moans  Is  a  poor  repre¬ 
sentation  of  the  relationship  between  an  Individual's  accuracy  of  assim¬ 
ilation  and  the  degree  of  certitude  he  ascribes  to  his  accuracy.  Hot 
only  Is  the  agreement  over  all  conditions  less  than  would  be  desired 
(r  ■  .62);  It  also  varies  considerably  as  a  function  of  enhancement 
technique,  only  double-cue  coding  showing  a  relationship  such  that  more 
than  50  percent  of  the  certitude  variance  can  be  accounted  for  by  accuracy 
variance.  Apparently,  the  less  conspicuous  the  changes  In  the  updated 
slides  the  less  the  subjects  were  aware  that  they  were  right  when  In  fact 
they  were  and  that  they  were  not  right  when  In  fact  they  were  not.  Also, 

It  Is  not  surprising  that  people  are  more  likely  to  be  100  percent  correct 
when  they  say  "absolutely  certain"  than  they  are  to  say  "absolutely  certain” 
when  they  are  100  percent  right.  The  fact  that  these  Judgments  concern  an 
end  category  of  the  continuum,  the  literal  meaning  of  the  anchor  terminol¬ 
ogy  itself— and  previous  findings  which  frequently  Indicate  conservatism  is 
such  Judgments— would  lead  to  this  expectation. 

The  major  efficacy  of  enhancement  techniques  Is  asstmwd  to  reside 
In  reducing  the  difficulty  of  the  task.  Thus,  a  further  degradation  In 
the  certitude-accuracy  relationship  within  an  enhancement  might  be  pro¬ 
duced  If  the  other  conditions— amount  and  extent  of  change— were  extended 
beyond  those  of  the  present  study  In  tense  of  making  the  task  more  diffi¬ 
cult.  Thus,  making  a  task  as  manageable  as  possible  by  provision  of  the 
best  possible  enhancement  aids  Is  clearly  advantageous. 

The  fact  that  the  earlier  findings  of  consistent  accuracy  of  assim¬ 
ilation  but  declining  certitude  for  Increasing  makers  of  "symbols  removed" 
were  not  sustained  In  the  present  study  suggests  subject  differences  and/ 
or  task  factors  as  determinants  of  this  relationship.  Task  realism  and 
research  designs  calling  for  repeated  measures  would  seem  to  be  desirable 
controls  for  future  research  In  this  area. 

Either  the  adjectival  anchors  on  the  certitude  continuum  assume 
soamvhet  different  meanings  for  the  subjects  as  a  function  of  enhance¬ 
ment  technique,  or  the  accuracy  of  application  varies  as  a  function  of 
enhancement  technique.  An  adequate  determination  of  the  nature  of  these 
effects  requires  a  continuum  where  a  quantitative  metric  Is  explicitly 
associated  with  each  category  or  subjects  are  allowed  free  response  In 
a  prescribed  quantitative  metric.  Studies  conducted  subsequent  to  the 
present  one  have  Incorporated  such  procedures  with  apparent  success. 
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Even  with  tbe  inadequacies  of  the  present  certitude  continuum,  indi¬ 
cations  are  that  to  tbe  extent  that  feelings  of  certitude  or  confidence 
influence  tbe  timeliness  and/or  appropriateness  of  decision,  discrep¬ 
ancies  of  tbe  present  Magnitude  between  accuracy  and  certitude  could 
frequently  have  a  deleterious  effect  on  tbe  decision  process. 
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13  •  ABSTRACT  continued 

condition.  Findings  indicate:  1)  The  more  effective  the  enhancement 
technique,  the  higher  the  certitude -accuracy  relationship.  With  the  best 
enhancement  technique  (double -cue  coding),  64  percent  of  the  certitude 
variance  could  be  accounted  for  by  accuracy  variance;  with  the  poorest 
(hard  copy),  only  20  percent.  2)  Both  over-certitude  and  under -certitude 
vas  evidenced,  with  over-certitude  tending  to  increase  with  the  less  ef¬ 
fective  enhancement  techniques.  3)  Increase  in  either  amount  of  informa¬ 
tion  presented  or  amount  of  updating  resulted  in  decline  in  both  mean 
accuracy  and  mean  certitude,  the  rate  varying  widely  over  the  different 
enhancement  techniques  and  over  types  of  update.  4)  Although  effects  of 
the  main  variables  on  accuracy  and  certitude  were  highly  similar,  the 
correspondence  did  not  hold  for  individual  performance  scores.  Findings 
suggest  need  to  improve  agreement  between  a  man's  performance  in  informa¬ 
tion  assimilation  and  his  Judgment  of  that  performance.  To  more  adequately 
determine  the  nature  of  the  effects  of  enhancement,  certitude  should  be 
measured  in  the  same  quantitative  metric  (percent)  as  accuracy. 
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